3D reconstruction is generally a difficult problem that involves capturing large amounts of data using large sensor arrays to obtain and process both depth and RGB data (or data in some other color space). Consequently, accurate generation of high fidelity geometric proxies for use in 3D reconstruction of objects and/or scenes is a complex process. Examples of 3D geometric proxies include, but are not limited to, depth maps, point based renderings, higher order geometric forms such as planes, objects, billboards, models, etc., and high fidelity proxies such as mesh based representations.
A “point cloud” is a set of vertices in a three-dimensional coordinate system that is typically used to represent the external surface of an object. Point clouds are often created by 3D scanners or other sensor arrays including various multi-camera array configurations for capturing 3D surface data. Point clouds are generally used for a variety of purposes, including the creation of geometric proxies by converting the point cloud to polygon or triangle geometric mesh models, NURBS surface models, or other types of geometric proxies using various 3D reconstruction techniques.
Unfortunately, when a point cloud representation of the scene or other object is reconstructed from captured data, there are often artifacts such as noisy or error prone stereo matches that erroneously extend particular boundaries or points within the scene, or of particular objects within the scene. Such artifacts generally lead to incorrect textures appearing on any 3D mesh surface or other geometric proxy derived from the point cloud. Further, where the data available for generating the point cloud is reduced or limited for any reason, the resulting sparsity of the point cloud typically further reduces the fidelity of the resulting geometric proxy.